Papers with visual dialog
GoG: Relation-aware Graph-over-Graph Network for Visual Dialog (2021.findings-acl)
Copied to clipboard
| Challenge: | Experimental results show that our model outperforms the strong baseline in both generative and discriminative settings by a significant margin. |
| Approach: | They propose a relation-aware graph-over-graph network (GoG) for visual dialog . their model outperforms the strong baseline in both generative and discriminative settings . |
| Outcome: | The proposed model outperforms baseline models in both generative and discriminative settings by a significant margin. |
CLEVR-Dialog: A Diagnostic Dataset for Multi-Round Reasoning in Visual Dialog (N19-1)
Copied to clipboard
| Challenge: | Visual Dialog is a multimodal task of answering a sequence of questions grounded in an image. |
| Approach: | They construct a dialog grammar that is grounded in the scene graphs of the images from the CLEVR dataset and use it to benchmark performance of standard visual dialog models. |
| Outcome: | The proposed model is based on a large diagnostic dataset for studying multi-round reasoning in visual dialog. |
Learning to Ground Visual Objects for Visual Dialog (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to ground visual objects are inadequate for visual dialog . a posterior distribution is inferred from context and questions, while posterior distributions are used to facilitate visual objects grounding. |
| Approach: | They propose a method to learn to ground visual objects for visual dialog using prior and posterior distributions over visual objects to facilitate visual objects grounding. |
| Outcome: | The proposed approach improves the existing models in generative and discriminative settings by a significant margin. |
Visual-Textual Alignment for Graph Inference in Visual Dialog (2020.coling-main)
Copied to clipboard
| Challenge: | Existing approaches to visual dialog do not understand semantic dependencies between visual and textual contents. |
| Approach: | They propose a Visual-Textual Alignment for Graph Inference network that makes up the lack of structural inference in visual dialog. |
| Outcome: | The proposed model outperforms existing models on a VisDial dataset. |
Region under Discussion for visual dialog (2021.emnlp-main)
Copied to clipboard
| Challenge: | Visual Dialog is assumed to require the dialog history to generate correct responses during a dialog. |
| Approach: | They propose an interpretable representation that visually grounds dialog history by constraining the image’s spatial features according to a semantic representation inspired by Question under Discussion. |
| Outcome: | The proposed representation constrains the image’s spatial features according to a semantic representation of the history inspired by the information structure notion of Question under Discussion. |
Multi-step Reasoning via Recurrent Dual Attention for Visual Dialog (P19-1)
Copied to clipboard
| Challenge: | Existing models for visual dialog infer the answer through multiple reasoning steps. |
| Approach: | They propose a model for visual dialog that uses multi-step reasoning to answer questions about an image. |
| Outcome: | The proposed model achieves a new state-of-the-art of 64.47% on the VisDial v1.0 dataset . |